{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "c5148957",
   "metadata": {},
   "source": [
    "# Categorical features"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "6da3874d",
   "metadata": {},
   "source": [
    "In the field of machine learning, categorical features play a crucial role in determining the predictive ability of a model. Categorical features are features that can take a limited number of values, such as color, gender or location. While these features can provide useful insights into patterns and relationships within data, they also present unique challenges for machine learning models.\n",
    "\n",
    "One of these challenges is the need to transform categorical features before they can be used by most models. This transformation involves converting categorical values into numerical values that can be processed by machine learning algorithms.\n",
    "\n",
    "Another challenge is dealing with infrequent categories, which can lead to biased models. If a categorical feature has a large number of categories, but some of them are rare or appear infrequently in the data, the model may not be able to learn accurately from these categories, resulting in biased predictions and inaccurate results.\n",
    "\n",
    "Despite these difficulties, categorical features are still an essential component in many use cases. When properly encoded and handled, machine learning models can effectively learn from patterns and relationships in categorical data, leading to better predictions.\n",
    "\n",
    "This document provides an overview of three of the most commonly used transformations: one-hot encoding, ordinal encoding, and target encoding. It explains how to apply them in the **skforecast** package using scikit-learn encoders, which provide a convenient and flexible way to pre-process data. It also shows how to use the native implementation of four popular gradient boosting frameworks – **XGBoost**, **LightGBM**, **scikit-learn's HistogramGradientBoosting** and **CatBoost** – to handle categorical features directly in the model.\n",
    "\n",
    "For a comprehensive demonstration of the use of categorical features in time series forecasting, check out the article [Forecasting time series with gradient boosting: Skforecast, XGBoost, LightGBM and CatBoost](https://www.cienciadedatos.net/documentos/py39-forecasting-time-series-with-skforecast-xgboost-lightgbm-catboost.html)."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "5bd0a8e9",
   "metadata": {},
   "source": [
    "<div class=\"admonition note\" name=\"html-admonition\" style=\"background: rgba(0,184,212,.1); padding-top: 0px; padding-bottom: 6px; border-radius: 8px; border-left: 8px solid #00b8d4; border-color: #00b8d4; padding-left: 10px; padding-right: 10px;\">\n",
    "\n",
    "<p class=\"title\">\n",
    "    <i style=\"font-size: 18px; color:#00b8d4;\"></i>\n",
    "    <b style=\"color: #00b8d4;\">&#9998 Note</b>\n",
    "</p>\n",
    "\n",
    "All of the transformations described in this document can be applied to the entire dataset, regardless of the forecaster. However, it is important to ensure that the transformations are learned only from the training data to avoid information leakage. Furthermore, the same transformation should be applied to the input data during prediction. To reduce the likelihood of errors and to ensure consistent application of the transformations, it is advisable to include the transformation within the forecaster object, so that it is handled internally.\n",
    "\n",
    "</div>"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "15dfe9dc",
   "metadata": {},
   "source": [
    "## Libraries and data\n",
    "\n",
    "The dataset used in this user guide consists of information on the number of users of a bicycle rental service, in addition to weather variables and holiday data. Two of the variables in the dataset, `holiday` and `weather`, are categorical."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "39a61044",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m\u001b[38;5;208mVersion scikit-learn: 1.6.1\n",
      "\u001b[1m\u001b[38;5;208mVersion lightgbm: 4.6.0\n",
      "\u001b[1m\u001b[38;5;208mVersion xgboost: 3.0.2\n"
     ]
    }
   ],
   "source": [
    "# Libraries\n",
    "# ==============================================================================\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import sklearn\n",
    "import lightgbm\n",
    "import xgboost\n",
    "from lightgbm import LGBMRegressor\n",
    "from sklearn.ensemble import HistGradientBoostingRegressor\n",
    "from xgboost import XGBRegressor\n",
    "from sklearn.preprocessing import OneHotEncoder\n",
    "from sklearn.preprocessing import OrdinalEncoder\n",
    "from sklearn.preprocessing import TargetEncoder\n",
    "from sklearn.preprocessing import FunctionTransformer\n",
    "from sklearn.compose import make_column_transformer\n",
    "from sklearn.compose import make_column_selector\n",
    "from sklearn.pipeline import make_pipeline\n",
    "from skforecast.datasets import fetch_dataset\n",
    "from skforecast.recursive import ForecasterRecursive\n",
    "plt.style.use('fivethirtyeight')\n",
    "plt.rcParams['lines.linewidth'] = 1.5\n",
    "color = '\\033[1m\\033[38;5;208m' \n",
    "print(f\"{color}Version scikit-learn: {sklearn.__version__}\")\n",
    "print(f\"{color}Version lightgbm: {lightgbm.__version__}\")\n",
    "print(f\"{color}Version xgboost: {xgboost.__version__}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "cc5647b4",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">╭───────────────────────────────── <span style=\"font-weight: bold\">bike_sharing</span> ──────────────────────────────────╮\n",
       "│ <span style=\"font-weight: bold\">Description:</span>                                                                    │\n",
       "│ Hourly usage of the bike share system in the city of Washington D.C. during the │\n",
       "│ years 2011 and 2012. In addition to the number of users per hour, information   │\n",
       "│ about weather conditions and holidays is available.                             │\n",
       "│                                                                                 │\n",
       "│ <span style=\"font-weight: bold\">Source:</span>                                                                         │\n",
       "│ Fanaee-T,Hadi. (2013). Bike Sharing Dataset. UCI Machine Learning Repository.   │\n",
       "│ https://doi.org/10.24432/C5W894.                                                │\n",
       "│                                                                                 │\n",
       "│ <span style=\"font-weight: bold\">URL:</span>                                                                            │\n",
       "│ https://raw.githubusercontent.com/skforecast/skforecast-                        │\n",
       "│ datasets/main/data/bike_sharing_dataset_clean.csv                               │\n",
       "│                                                                                 │\n",
       "│ <span style=\"font-weight: bold\">Shape:</span> 17544 rows x 12 columns                                                  │\n",
       "╰─────────────────────────────────────────────────────────────────────────────────╯\n",
       "</pre>\n"
      ],
      "text/plain": [
       "╭───────────────────────────────── \u001b[1mbike_sharing\u001b[0m ──────────────────────────────────╮\n",
       "│ \u001b[1mDescription:\u001b[0m                                                                    │\n",
       "│ Hourly usage of the bike share system in the city of Washington D.C. during the │\n",
       "│ years 2011 and 2012. In addition to the number of users per hour, information   │\n",
       "│ about weather conditions and holidays is available.                             │\n",
       "│                                                                                 │\n",
       "│ \u001b[1mSource:\u001b[0m                                                                         │\n",
       "│ Fanaee-T,Hadi. (2013). Bike Sharing Dataset. UCI Machine Learning Repository.   │\n",
       "│ https://doi.org/10.24432/C5W894.                                                │\n",
       "│                                                                                 │\n",
       "│ \u001b[1mURL:\u001b[0m                                                                            │\n",
       "│ https://raw.githubusercontent.com/skforecast/skforecast-                        │\n",
       "│ datasets/main/data/bike_sharing_dataset_clean.csv                               │\n",
       "│                                                                                 │\n",
       "│ \u001b[1mShape:\u001b[0m 17544 rows x 12 columns                                                  │\n",
       "╰─────────────────────────────────────────────────────────────────────────────────╯\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Downloading data\n",
    "# ==============================================================================\n",
    "data = fetch_dataset(name='bike_sharing', raw=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "holiday     object\n",
      "weather     object\n",
      "temp       float64\n",
      "hum        float64\n",
      "users      float64\n",
      "dtype: object\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>holiday</th>\n",
       "      <th>weather</th>\n",
       "      <th>temp</th>\n",
       "      <th>hum</th>\n",
       "      <th>users</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>date_time</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2011-01-01 00:00:00</th>\n",
       "      <td>0</td>\n",
       "      <td>clear</td>\n",
       "      <td>9.84</td>\n",
       "      <td>81.0</td>\n",
       "      <td>16.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2011-01-01 01:00:00</th>\n",
       "      <td>0</td>\n",
       "      <td>clear</td>\n",
       "      <td>9.02</td>\n",
       "      <td>80.0</td>\n",
       "      <td>40.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2011-01-01 02:00:00</th>\n",
       "      <td>0</td>\n",
       "      <td>clear</td>\n",
       "      <td>9.02</td>\n",
       "      <td>80.0</td>\n",
       "      <td>32.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                    holiday weather  temp   hum  users\n",
       "date_time                                             \n",
       "2011-01-01 00:00:00       0   clear  9.84  81.0   16.0\n",
       "2011-01-01 01:00:00       0   clear  9.02  80.0   40.0\n",
       "2011-01-01 02:00:00       0   clear  9.02  80.0   32.0"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Preprocess data\n",
    "# ==============================================================================\n",
    "data['date_time'] = pd.to_datetime(data['date_time'], format='%Y-%m-%d %H:%M:%S')\n",
    "data = data.set_index('date_time')\n",
    "data = data.asfreq('h')\n",
    "data = data.sort_index()\n",
    "data['holiday'] = data['holiday'].astype(int)\n",
    "data = data[['holiday', 'weather', 'temp', 'hum', 'users']]\n",
    "data[['holiday', 'weather']] = data[['holiday', 'weather']].astype(str)\n",
    "print(data.dtypes)\n",
    "data.head(3)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "42b58b7c",
   "metadata": {},
   "source": [
    "Only part of the data is used to simplify the example."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "5fb9a40a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Dates train : 2012-06-01 00:00:00 --- 2012-07-31 23:00:00  (n=1464)\n",
      "Dates test  : 2012-08-01 00:00:00 --- 2012-08-15 23:00:00  (n=360)\n"
     ]
    }
   ],
   "source": [
    "# Split train-test\n",
    "# ==============================================================================\n",
    "start_train = '2012-06-01 00:00:00'\n",
    "end_train = '2012-07-31 23:59:00'\n",
    "end_test = '2012-08-15 23:59:00'\n",
    "data_train = data.loc[start_train:end_train, :]\n",
    "data_test  = data.loc[end_train:end_test, :]\n",
    "\n",
    "print(\n",
    "    f\"Dates train : {data_train.index.min()} --- {data_train.index.max()}\"\n",
    "    f\"  (n={len(data_train)})\"\n",
    ")\n",
    "print(\n",
    "    f\"Dates test  : {data_test.index.min()} --- {data_test.index.max()}\"\n",
    "    f\"  (n={len(data_test)})\"\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "c879c7a7",
   "metadata": {},
   "source": [
    "## One Hot Encoding\n",
    "\n",
    "One hot encoding, also known as dummy encoding or one-of-K encoding, consists of replacing the categorical variable with a set of binary variables that take the value 0 or 1 to indicate whether a particular category is present in an observation. For example, suppose a dataset contains a categorical variable called \"color\" with the possible values of \"red,\" \"blue,\" and \"green\". Using one hot encoding, this variable is converted into three binary variables such as `color_red`, `color_blue`, and `color_green`, where each variable takes a value of 0 or 1 depending on the category.\n",
    "\n",
    "The [OneHotEncoder](https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.OneHotEncoder.html#sklearn.preprocessing.OneHotEncoder) class in scikit-learn can be used to transform any categorical feature with *n* possible values into *n* new binary features, where one of them takes the value 1, and all the others take the value 0. The `OneHotEncoder` can be configured to handle certain corner cases, including unknown categories, missing values, and infrequent categories.\n",
    "\n",
    "+ When `handle_unknown='ignore'` and `drop` is not `None`, unknown categories are encoded as zeros. Additionally, if a feature contains both `np.nan` and `None`, they are considered separate categories.\n",
    "\n",
    "+ It supports the aggregation of infrequent categories into a single output for each feature. The parameters to enable the aggregation of infrequent categories are `min_frequency` and `max_categories`. By setting `handle_unknown` to 'infrequent_if_exist', unknown categories are considered infrequent.\n",
    "\n",
    "+ To avoid collinearity between features, it is possible to drop one of the categories per feature using the `drop` argument. This is especially important when using linear models.\n",
    "\n",
    "\n",
    "[ColumnTransformers](https://scikit-learn.org/stable/modules/generated/sklearn.compose.ColumnTransformer.html) in scikit-learn provide a powerful way to define transformations and apply them to specific features. By encapsulating the `OneHotEncoder` in a `ColumnTransformer` object, it can be passed to a forecaster using the `transformer_exog` argument."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "a90de56c",
   "metadata": {},
   "outputs": [],
   "source": [
    "# ColumnTransformer with one-hot encoding\n",
    "# ==============================================================================\n",
    "# A ColumnTransformer is used to transform categorical features (no numerical)\n",
    "# using one-hot encoding. Numeric features are left untouched. For binary\n",
    "# features, only one column is created.\n",
    "one_hot_encoder = make_column_transformer(\n",
    "                      (\n",
    "                          OneHotEncoder(sparse_output=False, drop='if_binary'),\n",
    "                          make_column_selector(dtype_exclude=np.number)\n",
    "                      ),\n",
    "                      remainder=\"passthrough\",\n",
    "                      verbose_feature_names_out=False,\n",
    "                  ).set_output(transform=\"pandas\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "ad53b93d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "    <style>\n",
       "        .container-4b6221e309a446fab0a4fb9e2418aabc {\n",
       "            font-family: 'Arial', sans-serif;\n",
       "            font-size: 0.9em;\n",
       "            color: #333333;\n",
       "            border: 1px solid #ddd;\n",
       "            background-color: #f0f8ff;\n",
       "            padding: 5px 15px;\n",
       "            border-radius: 8px;\n",
       "            max-width: 600px;\n",
       "            #margin: auto;\n",
       "        }\n",
       "        .container-4b6221e309a446fab0a4fb9e2418aabc h2 {\n",
       "            font-size: 1.5em;\n",
       "            color: #222222;\n",
       "            border-bottom: 2px solid #ddd;\n",
       "            padding-bottom: 5px;\n",
       "            margin-bottom: 15px;\n",
       "            margin-top: 5px;\n",
       "        }\n",
       "        .container-4b6221e309a446fab0a4fb9e2418aabc details {\n",
       "            margin: 10px 0;\n",
       "        }\n",
       "        .container-4b6221e309a446fab0a4fb9e2418aabc summary {\n",
       "            font-weight: bold;\n",
       "            font-size: 1.1em;\n",
       "            color: #000000;\n",
       "            cursor: pointer;\n",
       "            margin-bottom: 5px;\n",
       "            background-color: #b3dbfd;\n",
       "            padding: 5px;\n",
       "            border-radius: 5px;\n",
       "        }\n",
       "        .container-4b6221e309a446fab0a4fb9e2418aabc summary:hover {\n",
       "            color: #000000;\n",
       "            background-color: #e0e0e0;\n",
       "        }\n",
       "        .container-4b6221e309a446fab0a4fb9e2418aabc ul {\n",
       "            font-family: 'Courier New', monospace;\n",
       "            list-style-type: none;\n",
       "            padding-left: 20px;\n",
       "            margin: 10px 0;\n",
       "            line-height: normal;\n",
       "        }\n",
       "        .container-4b6221e309a446fab0a4fb9e2418aabc li {\n",
       "            margin: 5px 0;\n",
       "            font-family: 'Courier New', monospace;\n",
       "        }\n",
       "        .container-4b6221e309a446fab0a4fb9e2418aabc li strong {\n",
       "            font-weight: bold;\n",
       "            color: #444444;\n",
       "        }\n",
       "        .container-4b6221e309a446fab0a4fb9e2418aabc li::before {\n",
       "            content: \"- \";\n",
       "            color: #666666;\n",
       "        }\n",
       "        .container-4b6221e309a446fab0a4fb9e2418aabc a {\n",
       "            color: #001633;\n",
       "            text-decoration: none;\n",
       "        }\n",
       "        .container-4b6221e309a446fab0a4fb9e2418aabc a:hover {\n",
       "            color: #359ccb; \n",
       "        }\n",
       "    </style>\n",
       "    \n",
       "        <div class=\"container-4b6221e309a446fab0a4fb9e2418aabc\">\n",
       "            <p style=\"font-size: 1.5em; font-weight: bold; margin-block-start: 0.83em; margin-block-end: 0.83em;\">ForecasterRecursive</p>\n",
       "            <details open>\n",
       "                <summary>General Information</summary>\n",
       "                <ul>\n",
       "                    <li><strong>Estimator:</strong> LGBMRegressor</li>\n",
       "                    <li><strong>Lags:</strong> [1 2 3 4 5]</li>\n",
       "                    <li><strong>Window features:</strong> None</li>\n",
       "                    <li><strong>Window size:</strong> 5</li>\n",
       "                    <li><strong>Series name:</strong> users</li>\n",
       "                    <li><strong>Exogenous included:</strong> True</li>\n",
       "                    <li><strong>Weight function included:</strong> False</li>\n",
       "                    <li><strong>Differentiation order:</strong> None</li>\n",
       "                    <li><strong>Creation date:</strong> 2025-11-26 14:43:14</li>\n",
       "                    <li><strong>Last fit date:</strong> 2025-11-26 14:43:15</li>\n",
       "                    <li><strong>Skforecast version:</strong> 0.19.0</li>\n",
       "                    <li><strong>Python version:</strong> 3.12.11</li>\n",
       "                    <li><strong>Forecaster id:</strong> None</li>\n",
       "                </ul>\n",
       "            </details>\n",
       "            <details>\n",
       "                <summary>Exogenous Variables</summary>\n",
       "                <ul>\n",
       "                    holiday, weather, temp, hum\n",
       "                </ul>\n",
       "            </details>\n",
       "            <details>\n",
       "                <summary>Data Transformations</summary>\n",
       "                <ul>\n",
       "                    <li><strong>Transformer for y:</strong> None</li>\n",
       "                    <li><strong>Transformer for exog:</strong> ColumnTransformer(remainder='passthrough',\n",
       "                  transformers=[('onehotencoder',\n",
       "                                 OneHotEncoder(drop='if_binary',\n",
       "                                               sparse_output=False),\n",
       "                                 <sklearn.compose._column_transformer.make_column_selector object at 0x000001C0E0A7BE00>)],\n",
       "                  verbose_feature_names_out=False)</li>\n",
       "                </ul>\n",
       "            </details>\n",
       "            <details>\n",
       "                <summary>Training Information</summary>\n",
       "                <ul>\n",
       "                    <li><strong>Training range:</strong> [Timestamp('2011-01-01 00:00:00'), Timestamp('2012-07-31 23:00:00')]</li>\n",
       "                    <li><strong>Training index type:</strong> DatetimeIndex</li>\n",
       "                    <li><strong>Training index frequency:</strong> <Hour></li>\n",
       "                </ul>\n",
       "            </details>\n",
       "            <details>\n",
       "                <summary>Estimator Parameters</summary>\n",
       "                <ul>\n",
       "                    {'boosting_type': 'gbdt', 'class_weight': None, 'colsample_bytree': 1.0, 'importance_type': 'split', 'learning_rate': 0.1, 'max_depth': -1, 'min_child_samples': 20, 'min_child_weight': 0.001, 'min_split_gain': 0.0, 'n_estimators': 100, 'n_jobs': None, 'num_leaves': 31, 'objective': None, 'random_state': 123, 'reg_alpha': 0.0, 'reg_lambda': 0.0, 'subsample': 1.0, 'subsample_for_bin': 200000, 'subsample_freq': 0, 'verbose': -1}\n",
       "                </ul>\n",
       "            </details>\n",
       "            <details>\n",
       "                <summary>Fit Kwargs</summary>\n",
       "                <ul>\n",
       "                    {}\n",
       "                </ul>\n",
       "            </details>\n",
       "            <p>\n",
       "                <a href=\"https://skforecast.org/0.19.0/api/forecasterrecursive.html\">&#128712 <strong>API Reference</strong></a>\n",
       "                &nbsp;&nbsp;\n",
       "                <a href=\"https://skforecast.org/0.19.0/user_guides/autoregressive-forecaster.html\">&#128462 <strong>User Guide</strong></a>\n",
       "            </p>\n",
       "        </div>\n",
       "        "
      ],
      "text/plain": [
       "=================== \n",
       "ForecasterRecursive \n",
       "=================== \n",
       "Estimator: LGBMRegressor \n",
       "Lags: [1 2 3 4 5] \n",
       "Window features: None \n",
       "Window size: 5 \n",
       "Series name: users \n",
       "Exogenous included: True \n",
       "Exogenous names: holiday, weather, temp, hum \n",
       "Transformer for y: None \n",
       "Transformer for exog: ColumnTransformer(remainder='passthrough',\n",
       "                  transformers=[('onehotencoder',\n",
       "                                 OneHotEncoder(drop='if_binary',\n",
       "                                               sparse_output=False),\n",
       "                                 <sklearn.compose._column_transformer.make_column_selector object at 0x000001C0E0A7BE00>)],\n",
       "                  verbose_feature_names_out=False) \n",
       "Weight function included: False \n",
       "Differentiation order: None \n",
       "Training range: [Timestamp('2011-01-01 00:00:00'), Timestamp('2012-07-31 23:00:00')] \n",
       "Training index type: DatetimeIndex \n",
       "Training index frequency: <Hour> \n",
       "Estimator parameters: \n",
       "    {'boosting_type': 'gbdt', 'class_weight': None, 'colsample_bytree': 1.0,\n",
       "    'importance_type': 'split', 'learning_rate': 0.1, 'max_depth': -1,\n",
       "    'min_child_samples': 20, 'min_child_weight': 0.001, 'min_split_gain': 0.0,\n",
       "    'n_estimators': 100, 'n_jobs': None, 'num_leaves': 31, 'objective': None,\n",
       "    'random_state': 123, 'reg_alpha': 0.0, 'reg_lambda': 0.0, 'subsample': 1.0,\n",
       "    'subsample_for_bin': 200000, 'subsample_freq': 0, 'verbose': -1} \n",
       "fit_kwargs: {} \n",
       "Creation date: 2025-11-26 14:43:14 \n",
       "Last fit date: 2025-11-26 14:43:15 \n",
       "Skforecast version: 0.19.0 \n",
       "Python version: 3.12.11 \n",
       "Forecaster id: None "
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create and fit forecaster with a transformer for exogenous features\n",
    "# ==============================================================================\n",
    "exog_features = ['holiday', 'weather', 'temp', 'hum']\n",
    "\n",
    "forecaster = ForecasterRecursive(\n",
    "                 estimator        = LGBMRegressor(random_state=123, verbose=-1),\n",
    "                 lags             = 5,\n",
    "                 transformer_exog = one_hot_encoder\n",
    "             )\n",
    "\n",
    "forecaster.fit(\n",
    "    y    = data.loc[:end_train, 'users'],\n",
    "    exog = data.loc[:end_train, exog_features]\n",
    ")\n",
    "\n",
    "forecaster"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "a2d4e38c",
   "metadata": {},
   "source": [
    "Once the forecaster has been trained, the transformer can be inspected (feature_names_in, feature_names_out, ...) by accessing the `transformer_exog` attribute."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "705f4d9c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['holiday_1' 'weather_clear' 'weather_mist' 'weather_rain' 'temp' 'hum']\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-1 {\n",
       "  /* Definition of color scheme common for light and dark mode */\n",
       "  --sklearn-color-text: #000;\n",
       "  --sklearn-color-text-muted: #666;\n",
       "  --sklearn-color-line: gray;\n",
       "  /* Definition of color scheme for unfitted estimators */\n",
       "  --sklearn-color-unfitted-level-0: #fff5e6;\n",
       "  --sklearn-color-unfitted-level-1: #f6e4d2;\n",
       "  --sklearn-color-unfitted-level-2: #ffe0b3;\n",
       "  --sklearn-color-unfitted-level-3: chocolate;\n",
       "  /* Definition of color scheme for fitted estimators */\n",
       "  --sklearn-color-fitted-level-0: #f0f8ff;\n",
       "  --sklearn-color-fitted-level-1: #d4ebff;\n",
       "  --sklearn-color-fitted-level-2: #b3dbfd;\n",
       "  --sklearn-color-fitted-level-3: cornflowerblue;\n",
       "\n",
       "  /* Specific color for light theme */\n",
       "  --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
       "  --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, white)));\n",
       "  --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
       "  --sklearn-color-icon: #696969;\n",
       "\n",
       "  @media (prefers-color-scheme: dark) {\n",
       "    /* Redefinition of color scheme for dark theme */\n",
       "    --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
       "    --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, #111)));\n",
       "    --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
       "    --sklearn-color-icon: #878787;\n",
       "  }\n",
       "}\n",
       "\n",
       "#sk-container-id-1 {\n",
       "  color: var(--sklearn-color-text);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 pre {\n",
       "  padding: 0;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 input.sk-hidden--visually {\n",
       "  border: 0;\n",
       "  clip: rect(1px 1px 1px 1px);\n",
       "  clip: rect(1px, 1px, 1px, 1px);\n",
       "  height: 1px;\n",
       "  margin: -1px;\n",
       "  overflow: hidden;\n",
       "  padding: 0;\n",
       "  position: absolute;\n",
       "  width: 1px;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-dashed-wrapped {\n",
       "  border: 1px dashed var(--sklearn-color-line);\n",
       "  margin: 0 0.4em 0.5em 0.4em;\n",
       "  box-sizing: border-box;\n",
       "  padding-bottom: 0.4em;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-container {\n",
       "  /* jupyter's `normalize.less` sets `[hidden] { display: none; }`\n",
       "     but bootstrap.min.css set `[hidden] { display: none !important; }`\n",
       "     so we also need the `!important` here to be able to override the\n",
       "     default hidden behavior on the sphinx rendered scikit-learn.org.\n",
       "     See: https://github.com/scikit-learn/scikit-learn/issues/21755 */\n",
       "  display: inline-block !important;\n",
       "  position: relative;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-text-repr-fallback {\n",
       "  display: none;\n",
       "}\n",
       "\n",
       "div.sk-parallel-item,\n",
       "div.sk-serial,\n",
       "div.sk-item {\n",
       "  /* draw centered vertical line to link estimators */\n",
       "  background-image: linear-gradient(var(--sklearn-color-text-on-default-background), var(--sklearn-color-text-on-default-background));\n",
       "  background-size: 2px 100%;\n",
       "  background-repeat: no-repeat;\n",
       "  background-position: center center;\n",
       "}\n",
       "\n",
       "/* Parallel-specific style estimator block */\n",
       "\n",
       "#sk-container-id-1 div.sk-parallel-item::after {\n",
       "  content: \"\";\n",
       "  width: 100%;\n",
       "  border-bottom: 2px solid var(--sklearn-color-text-on-default-background);\n",
       "  flex-grow: 1;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-parallel {\n",
       "  display: flex;\n",
       "  align-items: stretch;\n",
       "  justify-content: center;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  position: relative;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-parallel-item {\n",
       "  display: flex;\n",
       "  flex-direction: column;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-parallel-item:first-child::after {\n",
       "  align-self: flex-end;\n",
       "  width: 50%;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-parallel-item:last-child::after {\n",
       "  align-self: flex-start;\n",
       "  width: 50%;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-parallel-item:only-child::after {\n",
       "  width: 0;\n",
       "}\n",
       "\n",
       "/* Serial-specific style estimator block */\n",
       "\n",
       "#sk-container-id-1 div.sk-serial {\n",
       "  display: flex;\n",
       "  flex-direction: column;\n",
       "  align-items: center;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  padding-right: 1em;\n",
       "  padding-left: 1em;\n",
       "}\n",
       "\n",
       "\n",
       "/* Toggleable style: style used for estimator/Pipeline/ColumnTransformer box that is\n",
       "clickable and can be expanded/collapsed.\n",
       "- Pipeline and ColumnTransformer use this feature and define the default style\n",
       "- Estimators will overwrite some part of the style using the `sk-estimator` class\n",
       "*/\n",
       "\n",
       "/* Pipeline and ColumnTransformer style (default) */\n",
       "\n",
       "#sk-container-id-1 div.sk-toggleable {\n",
       "  /* Default theme specific background. It is overwritten whether we have a\n",
       "  specific estimator or a Pipeline/ColumnTransformer */\n",
       "  background-color: var(--sklearn-color-background);\n",
       "}\n",
       "\n",
       "/* Toggleable label */\n",
       "#sk-container-id-1 label.sk-toggleable__label {\n",
       "  cursor: pointer;\n",
       "  display: flex;\n",
       "  width: 100%;\n",
       "  margin-bottom: 0;\n",
       "  padding: 0.5em;\n",
       "  box-sizing: border-box;\n",
       "  text-align: center;\n",
       "  align-items: start;\n",
       "  justify-content: space-between;\n",
       "  gap: 0.5em;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 label.sk-toggleable__label .caption {\n",
       "  font-size: 0.6rem;\n",
       "  font-weight: lighter;\n",
       "  color: var(--sklearn-color-text-muted);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 label.sk-toggleable__label-arrow:before {\n",
       "  /* Arrow on the left of the label */\n",
       "  content: \"▸\";\n",
       "  float: left;\n",
       "  margin-right: 0.25em;\n",
       "  color: var(--sklearn-color-icon);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 label.sk-toggleable__label-arrow:hover:before {\n",
       "  color: var(--sklearn-color-text);\n",
       "}\n",
       "\n",
       "/* Toggleable content - dropdown */\n",
       "\n",
       "#sk-container-id-1 div.sk-toggleable__content {\n",
       "  max-height: 0;\n",
       "  max-width: 0;\n",
       "  overflow: hidden;\n",
       "  text-align: left;\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-toggleable__content.fitted {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-toggleable__content pre {\n",
       "  margin: 0.2em;\n",
       "  border-radius: 0.25em;\n",
       "  color: var(--sklearn-color-text);\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-toggleable__content.fitted pre {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 input.sk-toggleable__control:checked~div.sk-toggleable__content {\n",
       "  /* Expand drop-down */\n",
       "  max-height: 200px;\n",
       "  max-width: 100%;\n",
       "  overflow: auto;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {\n",
       "  content: \"▾\";\n",
       "}\n",
       "\n",
       "/* Pipeline/ColumnTransformer-specific style */\n",
       "\n",
       "#sk-container-id-1 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-label.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Estimator-specific style */\n",
       "\n",
       "/* Colorize estimator box */\n",
       "#sk-container-id-1 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-estimator.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-label label.sk-toggleable__label,\n",
       "#sk-container-id-1 div.sk-label label {\n",
       "  /* The background is the default theme color */\n",
       "  color: var(--sklearn-color-text-on-default-background);\n",
       "}\n",
       "\n",
       "/* On hover, darken the color of the background */\n",
       "#sk-container-id-1 div.sk-label:hover label.sk-toggleable__label {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "/* Label box, darken color on hover, fitted */\n",
       "#sk-container-id-1 div.sk-label.fitted:hover label.sk-toggleable__label.fitted {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Estimator label */\n",
       "\n",
       "#sk-container-id-1 div.sk-label label {\n",
       "  font-family: monospace;\n",
       "  font-weight: bold;\n",
       "  display: inline-block;\n",
       "  line-height: 1.2em;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-label-container {\n",
       "  text-align: center;\n",
       "}\n",
       "\n",
       "/* Estimator-specific */\n",
       "#sk-container-id-1 div.sk-estimator {\n",
       "  font-family: monospace;\n",
       "  border: 1px dotted var(--sklearn-color-border-box);\n",
       "  border-radius: 0.25em;\n",
       "  box-sizing: border-box;\n",
       "  margin-bottom: 0.5em;\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-estimator.fitted {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "/* on hover */\n",
       "#sk-container-id-1 div.sk-estimator:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-estimator.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Specification for estimator info (e.g. \"i\" and \"?\") */\n",
       "\n",
       "/* Common style for \"i\" and \"?\" */\n",
       "\n",
       ".sk-estimator-doc-link,\n",
       "a:link.sk-estimator-doc-link,\n",
       "a:visited.sk-estimator-doc-link {\n",
       "  float: right;\n",
       "  font-size: smaller;\n",
       "  line-height: 1em;\n",
       "  font-family: monospace;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  border-radius: 1em;\n",
       "  height: 1em;\n",
       "  width: 1em;\n",
       "  text-decoration: none !important;\n",
       "  margin-left: 0.5em;\n",
       "  text-align: center;\n",
       "  /* unfitted */\n",
       "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-unfitted-level-1);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link.fitted,\n",
       "a:link.sk-estimator-doc-link.fitted,\n",
       "a:visited.sk-estimator-doc-link.fitted {\n",
       "  /* fitted */\n",
       "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-fitted-level-1);\n",
       "}\n",
       "\n",
       "/* On hover */\n",
       "div.sk-estimator:hover .sk-estimator-doc-link:hover,\n",
       ".sk-estimator-doc-link:hover,\n",
       "div.sk-label-container:hover .sk-estimator-doc-link:hover,\n",
       ".sk-estimator-doc-link:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "div.sk-estimator.fitted:hover .sk-estimator-doc-link.fitted:hover,\n",
       ".sk-estimator-doc-link.fitted:hover,\n",
       "div.sk-label-container:hover .sk-estimator-doc-link.fitted:hover,\n",
       ".sk-estimator-doc-link.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "/* Span, style for the box shown on hovering the info icon */\n",
       ".sk-estimator-doc-link span {\n",
       "  display: none;\n",
       "  z-index: 9999;\n",
       "  position: relative;\n",
       "  font-weight: normal;\n",
       "  right: .2ex;\n",
       "  padding: .5ex;\n",
       "  margin: .5ex;\n",
       "  width: min-content;\n",
       "  min-width: 20ex;\n",
       "  max-width: 50ex;\n",
       "  color: var(--sklearn-color-text);\n",
       "  box-shadow: 2pt 2pt 4pt #999;\n",
       "  /* unfitted */\n",
       "  background: var(--sklearn-color-unfitted-level-0);\n",
       "  border: .5pt solid var(--sklearn-color-unfitted-level-3);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link.fitted span {\n",
       "  /* fitted */\n",
       "  background: var(--sklearn-color-fitted-level-0);\n",
       "  border: var(--sklearn-color-fitted-level-3);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link:hover span {\n",
       "  display: block;\n",
       "}\n",
       "\n",
       "/* \"?\"-specific style due to the `<a>` HTML tag */\n",
       "\n",
       "#sk-container-id-1 a.estimator_doc_link {\n",
       "  float: right;\n",
       "  font-size: 1rem;\n",
       "  line-height: 1em;\n",
       "  font-family: monospace;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  border-radius: 1rem;\n",
       "  height: 1rem;\n",
       "  width: 1rem;\n",
       "  text-decoration: none;\n",
       "  /* unfitted */\n",
       "  color: var(--sklearn-color-unfitted-level-1);\n",
       "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 a.estimator_doc_link.fitted {\n",
       "  /* fitted */\n",
       "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-fitted-level-1);\n",
       "}\n",
       "\n",
       "/* On hover */\n",
       "#sk-container-id-1 a.estimator_doc_link:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 a.estimator_doc_link.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-3);\n",
       "}\n",
       "</style><div id=\"sk-container-id-1\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>ColumnTransformer(remainder=&#x27;passthrough&#x27;,\n",
       "                  transformers=[(&#x27;onehotencoder&#x27;,\n",
       "                                 OneHotEncoder(drop=&#x27;if_binary&#x27;,\n",
       "                                               sparse_output=False),\n",
       "                                 &lt;sklearn.compose._column_transformer.make_column_selector object at 0x000001C0E0A7BE00&gt;)],\n",
       "                  verbose_feature_names_out=False)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item sk-dashed-wrapped\"><div class=\"sk-label-container\"><div class=\"sk-label fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-1\" type=\"checkbox\" ><label for=\"sk-estimator-id-1\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow\"><div><div>ColumnTransformer</div></div><div><a class=\"sk-estimator-doc-link fitted\" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.6/modules/generated/sklearn.compose.ColumnTransformer.html\">?<span>Documentation for ColumnTransformer</span></a><span class=\"sk-estimator-doc-link fitted\">i<span>Fitted</span></span></div></label><div class=\"sk-toggleable__content fitted\"><pre>ColumnTransformer(remainder=&#x27;passthrough&#x27;,\n",
       "                  transformers=[(&#x27;onehotencoder&#x27;,\n",
       "                                 OneHotEncoder(drop=&#x27;if_binary&#x27;,\n",
       "                                               sparse_output=False),\n",
       "                                 &lt;sklearn.compose._column_transformer.make_column_selector object at 0x000001C0E0A7BE00&gt;)],\n",
       "                  verbose_feature_names_out=False)</pre></div> </div></div><div class=\"sk-parallel\"><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-2\" type=\"checkbox\" ><label for=\"sk-estimator-id-2\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow\"><div><div>onehotencoder</div></div></label><div class=\"sk-toggleable__content fitted\"><pre>&lt;sklearn.compose._column_transformer.make_column_selector object at 0x000001C0E0A7BE00&gt;</pre></div> </div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-3\" type=\"checkbox\" ><label for=\"sk-estimator-id-3\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow\"><div><div>OneHotEncoder</div></div><div><a class=\"sk-estimator-doc-link fitted\" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.6/modules/generated/sklearn.preprocessing.OneHotEncoder.html\">?<span>Documentation for OneHotEncoder</span></a></div></label><div class=\"sk-toggleable__content fitted\"><pre>OneHotEncoder(drop=&#x27;if_binary&#x27;, sparse_output=False)</pre></div> </div></div></div></div></div><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-4\" type=\"checkbox\" ><label for=\"sk-estimator-id-4\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow\"><div><div>remainder</div></div></label><div class=\"sk-toggleable__content fitted\"><pre>[&#x27;temp&#x27;, &#x27;hum&#x27;]</pre></div> </div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-5\" type=\"checkbox\" ><label for=\"sk-estimator-id-5\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow\"><div><div>passthrough</div></div></label><div class=\"sk-toggleable__content fitted\"><pre>passthrough</pre></div> </div></div></div></div></div></div></div></div></div>"
      ],
      "text/plain": [
       "ColumnTransformer(remainder='passthrough',\n",
       "                  transformers=[('onehotencoder',\n",
       "                                 OneHotEncoder(drop='if_binary',\n",
       "                                               sparse_output=False),\n",
       "                                 <sklearn.compose._column_transformer.make_column_selector object at 0x000001C0E0A7BE00>)],\n",
       "                  verbose_feature_names_out=False)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Access to the transformer used for exogenous features\n",
    "# ==============================================================================\n",
    "print(forecaster.transformer_exog.get_feature_names_out())\n",
    "forecaster.transformer_exog"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "e1b232a6",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2012-08-01 00:00:00    88.946940\n",
       "2012-08-01 01:00:00    59.848451\n",
       "2012-08-01 02:00:00    28.870817\n",
       "Freq: h, Name: pred, dtype: float64"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Predictions\n",
    "# ==============================================================================\n",
    "forecaster.predict(steps=3, exog=data_test[exog_features])"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "f865c551",
   "metadata": {},
   "source": [
    "<div class=\"admonition note\" name=\"html-admonition\" style=\"background: rgba(0,184,212,.1); padding-top: 0px; padding-bottom: 6px; border-radius: 8px; border-left: 8px solid #00b8d4; border-color: #00b8d4; padding-left: 10px; padding-right: 10px;\">\n",
    "\n",
    "<p class=\"title\">\n",
    "    <i style=\"font-size: 18px; color:#00b8d4;\"></i>\n",
    "    <b style=\"color: #00b8d4;\">&#9998 Note</b>\n",
    "</p>\n",
    "\n",
    "It is possible to apply a transformation to the entire dataset independent of the forecaster. However, it is crucial to ensure that the transformations are only learned from the training data to avoid information leakage. In addition, the same transformation should be applied to the input data during prediction. It is therefore advisable to incorporate the transformation into the forecaster, so that it is handled internally. This approach ensures consistency in the application of transformations and reduces the likelihood of errors.\n",
    "\n",
    "To examine how data is being transformed, it is possible to use the <code>create_train_X_y()</code> method to generate the matrices used by the forecaster to train the model. This approach enables gaining insight into the specific data manipulations that occur during the training process.\n",
    "\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "69b6faa0",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "lag_1            float64\n",
      "lag_2            float64\n",
      "lag_3            float64\n",
      "lag_4            float64\n",
      "lag_5            float64\n",
      "holiday_1        float64\n",
      "weather_clear    float64\n",
      "weather_mist     float64\n",
      "weather_rain     float64\n",
      "temp             float64\n",
      "hum              float64\n",
      "dtype: object\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>lag_1</th>\n",
       "      <th>lag_2</th>\n",
       "      <th>lag_3</th>\n",
       "      <th>lag_4</th>\n",
       "      <th>lag_5</th>\n",
       "      <th>holiday_1</th>\n",
       "      <th>weather_clear</th>\n",
       "      <th>weather_mist</th>\n",
       "      <th>weather_rain</th>\n",
       "      <th>temp</th>\n",
       "      <th>hum</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>date_time</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2011-01-01 05:00:00</th>\n",
       "      <td>1.0</td>\n",
       "      <td>13.0</td>\n",
       "      <td>32.0</td>\n",
       "      <td>40.0</td>\n",
       "      <td>16.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>9.84</td>\n",
       "      <td>75.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2011-01-01 06:00:00</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>13.0</td>\n",
       "      <td>32.0</td>\n",
       "      <td>40.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>9.02</td>\n",
       "      <td>80.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2011-01-01 07:00:00</th>\n",
       "      <td>2.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>13.0</td>\n",
       "      <td>32.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>8.20</td>\n",
       "      <td>86.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2011-01-01 08:00:00</th>\n",
       "      <td>3.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>13.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>9.84</td>\n",
       "      <td>75.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2011-01-01 09:00:00</th>\n",
       "      <td>8.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>13.12</td>\n",
       "      <td>76.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                     lag_1  lag_2  lag_3  lag_4  lag_5  holiday_1  \\\n",
       "date_time                                                           \n",
       "2011-01-01 05:00:00    1.0   13.0   32.0   40.0   16.0        0.0   \n",
       "2011-01-01 06:00:00    1.0    1.0   13.0   32.0   40.0        0.0   \n",
       "2011-01-01 07:00:00    2.0    1.0    1.0   13.0   32.0        0.0   \n",
       "2011-01-01 08:00:00    3.0    2.0    1.0    1.0   13.0        0.0   \n",
       "2011-01-01 09:00:00    8.0    3.0    2.0    1.0    1.0        0.0   \n",
       "\n",
       "                     weather_clear  weather_mist  weather_rain   temp   hum  \n",
       "date_time                                                                    \n",
       "2011-01-01 05:00:00            0.0           1.0           0.0   9.84  75.0  \n",
       "2011-01-01 06:00:00            1.0           0.0           0.0   9.02  80.0  \n",
       "2011-01-01 07:00:00            1.0           0.0           0.0   8.20  86.0  \n",
       "2011-01-01 08:00:00            1.0           0.0           0.0   9.84  75.0  \n",
       "2011-01-01 09:00:00            1.0           0.0           0.0  13.12  76.0  "
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create training matrices\n",
    "# ==============================================================================\n",
    "X_train, y_train = forecaster.create_train_X_y(\n",
    "                       y    = data.loc[:end_train, 'users'],\n",
    "                       exog = data.loc[:end_train, exog_features]\n",
    "                   )\n",
    "\n",
    "print(X_train.dtypes)\n",
    "X_train.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "2e33abd5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>holiday_1</th>\n",
       "      <th>weather_clear</th>\n",
       "      <th>weather_mist</th>\n",
       "      <th>weather_rain</th>\n",
       "      <th>temp</th>\n",
       "      <th>hum</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>date_time</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2011-01-01 00:00:00</th>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>9.84</td>\n",
       "      <td>81.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2011-01-01 01:00:00</th>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>9.02</td>\n",
       "      <td>80.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2011-01-01 02:00:00</th>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>9.02</td>\n",
       "      <td>80.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2011-01-01 03:00:00</th>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>9.84</td>\n",
       "      <td>75.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2011-01-01 04:00:00</th>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>9.84</td>\n",
       "      <td>75.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                     holiday_1  weather_clear  weather_mist  weather_rain  \\\n",
       "date_time                                                                   \n",
       "2011-01-01 00:00:00        0.0            1.0           0.0           0.0   \n",
       "2011-01-01 01:00:00        0.0            1.0           0.0           0.0   \n",
       "2011-01-01 02:00:00        0.0            1.0           0.0           0.0   \n",
       "2011-01-01 03:00:00        0.0            1.0           0.0           0.0   \n",
       "2011-01-01 04:00:00        0.0            1.0           0.0           0.0   \n",
       "\n",
       "                     temp   hum  \n",
       "date_time                        \n",
       "2011-01-01 00:00:00  9.84  81.0  \n",
       "2011-01-01 01:00:00  9.02  80.0  \n",
       "2011-01-01 02:00:00  9.02  80.0  \n",
       "2011-01-01 03:00:00  9.84  75.0  \n",
       "2011-01-01 04:00:00  9.84  75.0  "
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Transform exogenous features using the transformer outside the forecaster\n",
    "# ==============================================================================\n",
    "exog_transformed = one_hot_encoder.fit_transform(data.loc[:end_train, exog_features])\n",
    "exog_transformed.head()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "2dea38b7",
   "metadata": {},
   "source": [
    "## Ordinal encoding\n",
    "\n",
    "Ordinal encoding is a technique used to convert categorical variables into numerical variables. Each category is assigned a unique numerical value based on its order or rank, as determined by a chosen criterion such as frequency or importance. This encoding method is particularly useful when categories have a natural order or ranking, such as educational qualifications. However, it is important to note that the numerical values assigned to each category do not represent any inherent numerical difference between them, but simply provide a numerical representation.\n",
    "\n",
    "The scikit-learn library provides the [OrdinalEncoder](https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.OrdinalEncoder.html) class, which allows users to replace categorical variables with ordinal numbers ranging from 0 to n_categories-1. In addition, this class includes the `encoded_missing_value` parameter, which allows for the encoding of missing values. It is important to note that this implementation arbitrarily assigns numbers to categories on a first-seen-first-served basis. Users should therefore exercise caution when interpreting the numerical values assigned to the categories. Other implementations, such as the [Feature-engine](https://feature-engine.trainindata.com/en/latest/user_guide/encoding/OrdinalEncoder.html), numbers can be ordered based on the mean of the target."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "0d4e30b2",
   "metadata": {},
   "outputs": [],
   "source": [
    "# ColumnTransformer with ordinal encoding\n",
    "# ==============================================================================\n",
    "# A ColumnTransformer is used to transform categorical (not numerical) features\n",
    "# using ordinal encoding. Numeric features are left untouched. Missing values\n",
    "# are coded as -1. If a new category is found in the test set, it is encoded\n",
    "# as -1.\n",
    "ordinal_encoder = make_column_transformer(\n",
    "                      (\n",
    "                          OrdinalEncoder(\n",
    "                              handle_unknown='use_encoded_value',\n",
    "                              unknown_value=-1,\n",
    "                              encoded_missing_value=-1\n",
    "                          ),\n",
    "                          make_column_selector(dtype_exclude=np.number)\n",
    "                      ),\n",
    "                      remainder=\"passthrough\",\n",
    "                      verbose_feature_names_out=False,\n",
    "                  ).set_output(transform=\"pandas\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "62742fa1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "    <style>\n",
       "        .container-e569e64e08d04e0f867fab08d514d59b {\n",
       "            font-family: 'Arial', sans-serif;\n",
       "            font-size: 0.9em;\n",
       "            color: #333333;\n",
       "            border: 1px solid #ddd;\n",
       "            background-color: #f0f8ff;\n",
       "            padding: 5px 15px;\n",
       "            border-radius: 8px;\n",
       "            max-width: 600px;\n",
       "            #margin: auto;\n",
       "        }\n",
       "        .container-e569e64e08d04e0f867fab08d514d59b h2 {\n",
       "            font-size: 1.5em;\n",
       "            color: #222222;\n",
       "            border-bottom: 2px solid #ddd;\n",
       "            padding-bottom: 5px;\n",
       "            margin-bottom: 15px;\n",
       "            margin-top: 5px;\n",
       "        }\n",
       "        .container-e569e64e08d04e0f867fab08d514d59b details {\n",
       "            margin: 10px 0;\n",
       "        }\n",
       "        .container-e569e64e08d04e0f867fab08d514d59b summary {\n",
       "            font-weight: bold;\n",
       "            font-size: 1.1em;\n",
       "            color: #000000;\n",
       "            cursor: pointer;\n",
       "            margin-bottom: 5px;\n",
       "            background-color: #b3dbfd;\n",
       "            padding: 5px;\n",
       "            border-radius: 5px;\n",
       "        }\n",
       "        .container-e569e64e08d04e0f867fab08d514d59b summary:hover {\n",
       "            color: #000000;\n",
       "            background-color: #e0e0e0;\n",
       "        }\n",
       "        .container-e569e64e08d04e0f867fab08d514d59b ul {\n",
       "            font-family: 'Courier New', monospace;\n",
       "            list-style-type: none;\n",
       "            padding-left: 20px;\n",
       "            margin: 10px 0;\n",
       "            line-height: normal;\n",
       "        }\n",
       "        .container-e569e64e08d04e0f867fab08d514d59b li {\n",
       "            margin: 5px 0;\n",
       "            font-family: 'Courier New', monospace;\n",
       "        }\n",
       "        .container-e569e64e08d04e0f867fab08d514d59b li strong {\n",
       "            font-weight: bold;\n",
       "            color: #444444;\n",
       "        }\n",
       "        .container-e569e64e08d04e0f867fab08d514d59b li::before {\n",
       "            content: \"- \";\n",
       "            color: #666666;\n",
       "        }\n",
       "        .container-e569e64e08d04e0f867fab08d514d59b a {\n",
       "            color: #001633;\n",
       "            text-decoration: none;\n",
       "        }\n",
       "        .container-e569e64e08d04e0f867fab08d514d59b a:hover {\n",
       "            color: #359ccb; \n",
       "        }\n",
       "    </style>\n",
       "    \n",
       "        <div class=\"container-e569e64e08d04e0f867fab08d514d59b\">\n",
       "            <p style=\"font-size: 1.5em; font-weight: bold; margin-block-start: 0.83em; margin-block-end: 0.83em;\">ForecasterRecursive</p>\n",
       "            <details open>\n",
       "                <summary>General Information</summary>\n",
       "                <ul>\n",
       "                    <li><strong>Estimator:</strong> LGBMRegressor</li>\n",
       "                    <li><strong>Lags:</strong> [1 2 3 4 5]</li>\n",
       "                    <li><strong>Window features:</strong> None</li>\n",
       "                    <li><strong>Window size:</strong> 5</li>\n",
       "                    <li><strong>Series name:</strong> users</li>\n",
       "                    <li><strong>Exogenous included:</strong> True</li>\n",
       "                    <li><strong>Weight function included:</strong> False</li>\n",
       "                    <li><strong>Differentiation order:</strong> None</li>\n",
       "                    <li><strong>Creation date:</strong> 2025-11-26 14:43:15</li>\n",
       "                    <li><strong>Last fit date:</strong> 2025-11-26 14:43:15</li>\n",
       "                    <li><strong>Skforecast version:</strong> 0.19.0</li>\n",
       "                    <li><strong>Python version:</strong> 3.12.11</li>\n",
       "                    <li><strong>Forecaster id:</strong> None</li>\n",
       "                </ul>\n",
       "            </details>\n",
       "            <details>\n",
       "                <summary>Exogenous Variables</summary>\n",
       "                <ul>\n",
       "                    holiday, weather, temp, hum\n",
       "                </ul>\n",
       "            </details>\n",
       "            <details>\n",
       "                <summary>Data Transformations</summary>\n",
       "                <ul>\n",
       "                    <li><strong>Transformer for y:</strong> None</li>\n",
       "                    <li><strong>Transformer for exog:</strong> ColumnTransformer(remainder='passthrough',\n",
       "                  transformers=[('ordinalencoder',\n",
       "                                 OrdinalEncoder(encoded_missing_value=-1,\n",
       "                                                handle_unknown='use_encoded_value',\n",
       "                                                unknown_value=-1),\n",
       "                                 <sklearn.compose._column_transformer.make_column_selector object at 0x000001C0E0F530B0>)],\n",
       "                  verbose_feature_names_out=False)</li>\n",
       "                </ul>\n",
       "            </details>\n",
       "            <details>\n",
       "                <summary>Training Information</summary>\n",
       "                <ul>\n",
       "                    <li><strong>Training range:</strong> [Timestamp('2011-01-01 00:00:00'), Timestamp('2012-07-31 23:00:00')]</li>\n",
       "                    <li><strong>Training index type:</strong> DatetimeIndex</li>\n",
       "                    <li><strong>Training index frequency:</strong> <Hour></li>\n",
       "                </ul>\n",
       "            </details>\n",
       "            <details>\n",
       "                <summary>Estimator Parameters</summary>\n",
       "                <ul>\n",
       "                    {'boosting_type': 'gbdt', 'class_weight': None, 'colsample_bytree': 1.0, 'importance_type': 'split', 'learning_rate': 0.1, 'max_depth': -1, 'min_child_samples': 20, 'min_child_weight': 0.001, 'min_split_gain': 0.0, 'n_estimators': 100, 'n_jobs': None, 'num_leaves': 31, 'objective': None, 'random_state': 123, 'reg_alpha': 0.0, 'reg_lambda': 0.0, 'subsample': 1.0, 'subsample_for_bin': 200000, 'subsample_freq': 0, 'verbose': -1}\n",
       "                </ul>\n",
       "            </details>\n",
       "            <details>\n",
       "                <summary>Fit Kwargs</summary>\n",
       "                <ul>\n",
       "                    {}\n",
       "                </ul>\n",
       "            </details>\n",
       "            <p>\n",
       "                <a href=\"https://skforecast.org/0.19.0/api/forecasterrecursive.html\">&#128712 <strong>API Reference</strong></a>\n",
       "                &nbsp;&nbsp;\n",
       "                <a href=\"https://skforecast.org/0.19.0/user_guides/autoregressive-forecaster.html\">&#128462 <strong>User Guide</strong></a>\n",
       "            </p>\n",
       "        </div>\n",
       "        "
      ],
      "text/plain": [
       "=================== \n",
       "ForecasterRecursive \n",
       "=================== \n",
       "Estimator: LGBMRegressor \n",
       "Lags: [1 2 3 4 5] \n",
       "Window features: None \n",
       "Window size: 5 \n",
       "Series name: users \n",
       "Exogenous included: True \n",
       "Exogenous names: holiday, weather, temp, hum \n",
       "Transformer for y: None \n",
       "Transformer for exog: ColumnTransformer(remainder='passthrough',\n",
       "                  transformers=[('ordinalencoder',\n",
       "                                 OrdinalEncoder(encoded_missing_value=-1,\n",
       "                                                handle_unknown='use_encoded_value',\n",
       "                                                unknown_value=-1),\n",
       "                                 <sklearn.compose._column_transformer.make_column_selector object at 0x000001C0E0F530B0>)],\n",
       "                  verbose_feature_names_out=False) \n",
       "Weight function included: False \n",
       "Differentiation order: None \n",
       "Training range: [Timestamp('2011-01-01 00:00:00'), Timestamp('2012-07-31 23:00:00')] \n",
       "Training index type: DatetimeIndex \n",
       "Training index frequency: <Hour> \n",
       "Estimator parameters: \n",
       "    {'boosting_type': 'gbdt', 'class_weight': None, 'colsample_bytree': 1.0,\n",
       "    'importance_type': 'split', 'learning_rate': 0.1, 'max_depth': -1,\n",
       "    'min_child_samples': 20, 'min_child_weight': 0.001, 'min_split_gain': 0.0,\n",
       "    'n_estimators': 100, 'n_jobs': None, 'num_leaves': 31, 'objective': None,\n",
       "    'random_state': 123, 'reg_alpha': 0.0, 'reg_lambda': 0.0, 'subsample': 1.0,\n",
       "    'subsample_for_bin': 200000, 'subsample_freq': 0, 'verbose': -1} \n",
       "fit_kwargs: {} \n",
       "Creation date: 2025-11-26 14:43:15 \n",
       "Last fit date: 2025-11-26 14:43:15 \n",
       "Skforecast version: 0.19.0 \n",
       "Python version: 3.12.11 \n",
       "Forecaster id: None "
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create and fit a forecaster with a transformer for exogenous features\n",
    "# ==============================================================================\n",
    "exog_features = ['holiday', 'weather', 'temp', 'hum']\n",
    "\n",
    "forecaster = ForecasterRecursive(\n",
    "                 estimator        = LGBMRegressor(random_state=123, verbose=-1),\n",
    "                 lags             = 5,\n",
    "                 transformer_exog = ordinal_encoder\n",
    "             )\n",
    "\n",
    "forecaster.fit(\n",
    "    y    = data.loc[:end_train, 'users'],\n",
    "    exog = data.loc[:end_train, exog_features]\n",
    ")\n",
    "\n",
    "forecaster"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "77fce7a5",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "lag_1      float64\n",
      "lag_2      float64\n",
      "lag_3      float64\n",
      "lag_4      float64\n",
      "lag_5      float64\n",
      "holiday    float64\n",
      "weather    float64\n",
      "temp       float64\n",
      "hum        float64\n",
      "dtype: object\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>lag_1</th>\n",
       "      <th>lag_2</th>\n",
       "      <th>lag_3</th>\n",
       "      <th>lag_4</th>\n",
       "      <th>lag_5</th>\n",
       "      <th>holiday</th>\n",
       "      <th>weather</th>\n",
       "      <th>temp</th>\n",
       "      <th>hum</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>date_time</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2011-01-01 05:00:00</th>\n",
       "      <td>1.0</td>\n",
       "      <td>13.0</td>\n",
       "      <td>32.0</td>\n",
       "      <td>40.0</td>\n",
       "      <td>16.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>9.84</td>\n",
       "      <td>75.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2011-01-01 06:00:00</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>13.0</td>\n",
       "      <td>32.0</td>\n",
       "      <td>40.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>9.02</td>\n",
       "      <td>80.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2011-01-01 07:00:00</th>\n",
       "      <td>2.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>13.0</td>\n",
       "      <td>32.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>8.20</td>\n",
       "      <td>86.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2011-01-01 08:00:00</th>\n",
       "      <td>3.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>13.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>9.84</td>\n",
       "      <td>75.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2011-01-01 09:00:00</th>\n",
       "      <td>8.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>13.12</td>\n",
       "      <td>76.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                     lag_1  lag_2  lag_3  lag_4  lag_5  holiday  weather  \\\n",
       "date_time                                                                  \n",
       "2011-01-01 05:00:00    1.0   13.0   32.0   40.0   16.0      0.0      1.0   \n",
       "2011-01-01 06:00:00    1.0    1.0   13.0   32.0   40.0      0.0      0.0   \n",
       "2011-01-01 07:00:00    2.0    1.0    1.0   13.0   32.0      0.0      0.0   \n",
       "2011-01-01 08:00:00    3.0    2.0    1.0    1.0   13.0      0.0      0.0   \n",
       "2011-01-01 09:00:00    8.0    3.0    2.0    1.0    1.0      0.0      0.0   \n",
       "\n",
       "                      temp   hum  \n",
       "date_time                         \n",
       "2011-01-01 05:00:00   9.84  75.0  \n",
       "2011-01-01 06:00:00   9.02  80.0  \n",
       "2011-01-01 07:00:00   8.20  86.0  \n",
       "2011-01-01 08:00:00   9.84  75.0  \n",
       "2011-01-01 09:00:00  13.12  76.0  "
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create training matrices\n",
    "# ==============================================================================\n",
    "X_train, y_train = forecaster.create_train_X_y(\n",
    "                       y    = data.loc[:end_train, 'users'],\n",
    "                       exog = data.loc[:end_train, exog_features]\n",
    "                   )\n",
    "\n",
    "print(X_train.dtypes)\n",
    "X_train.head()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "ddca9a92",
   "metadata": {},
   "source": [
    "Once the forecaster has been trained, the transformer can be inspected by accessing the `transformer_exog` attribute."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "503e22e7",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2012-08-01 00:00:00    89.096098\n",
       "2012-08-01 01:00:00    57.749964\n",
       "2012-08-01 02:00:00    29.263922\n",
       "Freq: h, Name: pred, dtype: float64"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Predictions\n",
    "# ==============================================================================\n",
    "forecaster.predict(steps=3, exog=data_test[exog_features])"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "8c5af312",
   "metadata": {},
   "source": [
    "## Target encoding\n",
    "\n",
    "Target encoding is a technic that encodes categorical variables based on the relationship between the categories and the target variable. Each category is encoded based on a shrinked estimate of the average target values for observations belonging to the category. The encoding scheme mixes the global target mean with the target mean conditioned on the value of the category.\n",
    "\n",
    "For example, suppose a categorical variable \"City\" with categories \"New York,\" \"Los Angeles,\" and \"Chicago,\" and a target variable \"Salary.\" One can calculate the mean salary for each city based on the training data, and use these mean values to encode the categories.\n",
    "\n",
    "This encoding scheme is useful with categorical features with high cardinality, where one-hot encoding would inflate the feature space making it more expensive for a downstream model to process. A classical example of high cardinality categories is location-based such as zip code or region.\n",
    "\n",
    "The [TargetEncoder](https://scikit-learn.org/dev/modules/generated/sklearn.preprocessing.TargetEncoder.html) class is available in Scikit-learn (since version 1.3). `TargetEncoder` considers missing values, such as `np.nan` or `None`, as another category and encodes them like any other category. Categories that are not seen during fit are encoded with the target mean, i.e. `target_mean_`. A more detailed description of target encoding can be found in the [scikit-learn user guide](https://scikit-learn.org/stable/modules/preprocessing.html#target-encoder)."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "45b401ba",
   "metadata": {},
   "source": [
    "<div class=\"admonition note\" name=\"html-admonition\" style=\"background: rgba(255,145,0,.1); padding-top: 0px; padding-bottom: 6px; border-radius: 8px; border-left: 8px solid #ff9100; border-color: #ff9100; padding-left: 10px; padding-right: 10px\">\n",
    "\n",
    "<p class=\"title\">\n",
    "    <i style=\"font-size: 18px; color:#ff9100; border-color: #ff1744;\"></i>\n",
    "    <b style=\"color: #ff9100;\"> <span style=\"color: #ff9100;\">&#9888;</span> Warning</b>\n",
    "</p>\n",
    "\n",
    "TargetEncoder differs from the other transformers in scikit-learn in that it requires not only the features to be transformed but also the response variable (target), in the context of prediction, this is the time series. Currently, the only transformers allowed in the prediction classes are those that do not require the target variable to be fitted. Therefore, to use target encoding, transformations must be applied outside the Forecaster object.\n",
    "\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "# ColumnTransformer with target encoding\n",
    "# ==============================================================================\n",
    "# A ColumnTransformer is used to transform categorical (not numerical) features\n",
    "# using target encoding. Numeric features are left untouched. TargetEncoder\n",
    "# considers missing values, such as np.nan or None, as another category and\n",
    "# encodes them like any other category. Categories that are not seen during fit\n",
    "# are encoded with the target mean\n",
    "\n",
    "target_encoder = make_column_transformer(\n",
    "                     (\n",
    "                         TargetEncoder(\n",
    "                             categories   = 'auto',\n",
    "                             target_type  = 'continuous',\n",
    "                             smooth       = 'auto',\n",
    "                             random_state = 9874\n",
    "                         ),\n",
    "                         make_column_selector(dtype_exclude=np.number)\n",
    "                     ),\n",
    "                     remainder=\"passthrough\",\n",
    "                     verbose_feature_names_out=False,\n",
    "                 ).set_output(transform=\"pandas\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>holiday</th>\n",
       "      <th>weather</th>\n",
       "      <th>temp</th>\n",
       "      <th>hum</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>date_time</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2011-01-01 00:00:00</th>\n",
       "      <td>172.823951</td>\n",
       "      <td>188.121327</td>\n",
       "      <td>9.84</td>\n",
       "      <td>81.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2011-01-01 01:00:00</th>\n",
       "      <td>172.607889</td>\n",
       "      <td>187.330734</td>\n",
       "      <td>9.02</td>\n",
       "      <td>80.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2011-01-01 02:00:00</th>\n",
       "      <td>173.476675</td>\n",
       "      <td>189.423278</td>\n",
       "      <td>9.02</td>\n",
       "      <td>80.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2011-01-01 03:00:00</th>\n",
       "      <td>172.823951</td>\n",
       "      <td>188.121327</td>\n",
       "      <td>9.84</td>\n",
       "      <td>75.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2011-01-01 04:00:00</th>\n",
       "      <td>172.823951</td>\n",
       "      <td>188.121327</td>\n",
       "      <td>9.84</td>\n",
       "      <td>75.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                        holiday     weather  temp   hum\n",
       "date_time                                              \n",
       "2011-01-01 00:00:00  172.823951  188.121327  9.84  81.0\n",
       "2011-01-01 01:00:00  172.607889  187.330734  9.02  80.0\n",
       "2011-01-01 02:00:00  173.476675  189.423278  9.02  80.0\n",
       "2011-01-01 03:00:00  172.823951  188.121327  9.84  75.0\n",
       "2011-01-01 04:00:00  172.823951  188.121327  9.84  75.0"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Transform the exogenous features using the transformer outside the forecaster\n",
    "# ==============================================================================\n",
    "exog_transformed = target_encoder.fit_transform(\n",
    "                       X = data.loc[:end_train, exog_features],\n",
    "                       y = data.loc[:end_train, 'users']\n",
    "                   )\n",
    "exog_transformed.head()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "a3b388bf",
   "metadata": {},
   "source": [
    "## Native implementation for categorical features\n",
    "\n",
    "Some machine learning models, including **XGBoost**, **LightGBM**, **CatBoost**, and **HistGradientBoostingRegressor**, provide built-in methods to handle categorical features, but they assume that the input categories are integers starting from 0 up to the number of categories [0, 1, ..., n_categories-1]. In practice, categorical variables are not coded with numbers but with strings, so an intermediate transformation step is necessary. Two options are:\n",
    "\n",
    "+ Set columns with categorical variables to the type `category`. For each column, the data structure consists of an array of categories and an array of integer values (codes) that point to the actual value of the array of categories. That is, internally it is a numeric array with a mapping that relates each value to a category. Models are able to automatically identify the columns of type `category` and access their internal codes.\n",
    "\n",
    "+ Preprocess the categorical columns with an `OrdinalEncoder` to transform their values to integers and explicitly indicate that the columns should be treated as categorical."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "935caed0",
   "metadata": {},
   "source": [
    "<div class=\"admonition note\" name=\"html-admonition\" style=\"background: rgba(255,145,0,.1); padding-top: 0px; padding-bottom: 6px; border-radius: 8px; border-left: 8px solid #ff9100; border-color: #ff9100; padding-left: 10px; padding-right: 10px\">\n",
    "\n",
    "<p class=\"title\">\n",
    "    <i style=\"font-size: 18px; color:#ff9100; border-color: #ff1744;\"></i>\n",
    "    <b style=\"color: #ff9100;\"> <span style=\"color: #ff9100;\">&#9888;</span> Warning</b>\n",
    "</p>\n",
    "\n",
    "When deploying models in production, it is strongly recommended to <b>avoid using automatic detection</b> based on pandas <code>category</code> type columns. Although pandas provides an internal coding for these columns, it is not consistent across different datasets and may vary depending on the categories present in each one. It is therefore crucial to be aware of this issue and to take appropriate measures to ensure consistency in the coding of categorical features when deploying models in production.\n",
    "\n",
    "At the time of writing, the authors have observed that <code>LightGBM</code> and <code>HistGradientBoostingRegressor</code> internally manage changes in the coding of categories to ensure consistency.\n",
    "\n",
    "If the user still wishes to rely on automatic detection of categorical features based on pandas data types, categorical variables must first be encoded as integers (ordinal encoding) and then stored as category type. This is necessary because skforecast uses a numeric numpy array internally to speed up the calculation.\n",
    "\n",
    "</div>"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "c5c109b9",
   "metadata": {},
   "source": [
    "## LightGBM"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "412e83ff",
   "metadata": {},
   "source": [
    "**Encoding the categories as integers and explicitly specifying the names of the categorical features**\n",
    "\n",
    "When creating a forecaster with `LGBMRegressor`, it is necessary to specify the names of the categorical columns using the `fit_kwargs` argument. This is because the `categorical_feature` argument is only specified in the `fit` method of `LGBMRegressor`, and not during its initialization."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "767c0dc1",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Transformer: ordinal encoding\n",
    "# ==============================================================================\n",
    "# A ColumnTransformer is used to transform categorical (not numerical) features\n",
    "# using ordinal encoding. Numeric features are left untouched. Missing values\n",
    "# are coded as -1. If a new category is found in the test set, it is encoded\n",
    "# as -1.\n",
    "categorical_features = data.select_dtypes(exclude=[np.number]).columns.tolist()\n",
    "transformer_exog = make_column_transformer(\n",
    "                       (\n",
    "                           OrdinalEncoder(\n",
    "                               dtype=int,\n",
    "                               handle_unknown=\"use_encoded_value\",\n",
    "                               unknown_value=-1,\n",
    "                               encoded_missing_value=-1\n",
    "                           ),\n",
    "                           categorical_features\n",
    "                       ),\n",
    "                       remainder=\"passthrough\",\n",
    "                       verbose_feature_names_out=False,\n",
    "                   ).set_output(transform=\"pandas\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "5eb9ac37",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create and fit forecaster indicating the categorical features\n",
    "# ==============================================================================\n",
    "exog_features = ['holiday', 'weather', 'temp', 'hum']\n",
    "forecaster = ForecasterRecursive(\n",
    "                 estimator        = LGBMRegressor(random_state=963, verbose=-1),\n",
    "                 lags             = 5,\n",
    "                 transformer_exog = transformer_exog,\n",
    "                 fit_kwargs       = {'categorical_feature': categorical_features}\n",
    "             )\n",
    " \n",
    "forecaster.fit(\n",
    "    y    = data.loc[:end_train, 'users'],\n",
    "    exog = data.loc[:end_train, exog_features]\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "a2204eb4",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2012-08-01 00:00:00    88.946940\n",
       "2012-08-01 01:00:00    59.848451\n",
       "2012-08-01 02:00:00    28.870817\n",
       "Freq: h, Name: pred, dtype: float64"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Predictions\n",
    "# ==============================================================================\n",
    "forecaster.predict(steps=3, exog=data_test[exog_features])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "2a32bf97",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['holiday', 'weather']"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Print the features considered as categorical by the forecaster\n",
    "# ==============================================================================\n",
    "estimator = forecaster.estimator\n",
    "cat_index = estimator.booster_.params.get('categorical_column')\n",
    "if cat_index is not None:\n",
    "    features_in_model = estimator.booster_.feature_name()\n",
    "    cat_features_in_model = [features_in_model[i] for i in cat_index]\n",
    "\n",
    "cat_features_in_model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "73c3516e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Feature 'holiday' categories and codes:\n",
      "  0: 0\n",
      "  1: 1\n",
      "Feature 'weather' categories and codes:\n",
      "  clear: 0\n",
      "  mist: 1\n",
      "  rain: 2\n"
     ]
    }
   ],
   "source": [
    "# Show the encoding applied to the categorical features\n",
    "# ==============================================================================\n",
    "ordinal_encoder = transformer_exog.named_transformers_['ordinalencoder']\n",
    "for feature, cats in zip(categorical_features, ordinal_encoder.categories_):\n",
    "    print(f\"Feature '{feature}' categories and codes:\")\n",
    "    for code, category in enumerate(cats):\n",
    "        print(f\"  {category}: {code}\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "60310d05",
   "metadata": {},
   "source": [
    "**Allow the model to automatically detect categorical features**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "f971a6e2",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Transformer: ordinal encoding and cast to category type\n",
    "# ==============================================================================\n",
    "# A ColumnTransformer is used to transform categorical (not numerical) features\n",
    "# using ordinal encoding. Numeric features are left untouched. Missing values\n",
    "# are coded as -1. If a new category is found in the test set, it is encoded\n",
    "# as -1. After encoding, the features are converted back to category type so that \n",
    "# they can be identified as categorical features by the estimator.\n",
    "\n",
    "pipeline_categorical = make_pipeline(\n",
    "                           OrdinalEncoder(\n",
    "                               dtype=int,\n",
    "                               handle_unknown=\"use_encoded_value\",\n",
    "                               unknown_value=-1,\n",
    "                               encoded_missing_value=-1\n",
    "                           ),\n",
    "                           FunctionTransformer(\n",
    "                               func=lambda x: x.astype('category'),\n",
    "                               feature_names_out= 'one-to-one'\n",
    "                           )\n",
    "                       )\n",
    "\n",
    "transformer_exog = make_column_transformer(\n",
    "                       (\n",
    "                           pipeline_categorical,\n",
    "                           make_column_selector(dtype_exclude=np.number)\n",
    "                       ),\n",
    "                       remainder=\"passthrough\",\n",
    "                       verbose_feature_names_out=False,\n",
    "                   ).set_output(transform=\"pandas\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "76d8f643",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create and fit forecaster with automatic detection of categorical features\n",
    "# ==============================================================================\n",
    "exog_features = ['holiday', 'weather', 'temp', 'hum']\n",
    "forecaster = ForecasterRecursive(\n",
    "                 estimator        = LGBMRegressor(random_state=963, verbose=-1),\n",
    "                 lags             = 5,\n",
    "                 transformer_exog = transformer_exog,\n",
    "                 fit_kwargs       = {'categorical_feature': 'auto'}\n",
    "             )\n",
    "            \n",
    "forecaster.fit(\n",
    "    y    = data.loc[:end_train, 'users'],\n",
    "    exog = data.loc[:end_train, exog_features]\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "869e0946",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2012-08-01 00:00:00    88.946940\n",
       "2012-08-01 01:00:00    59.848451\n",
       "2012-08-01 02:00:00    28.870817\n",
       "Freq: h, Name: pred, dtype: float64"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Predictions\n",
    "# ==============================================================================\n",
    "forecaster.predict(steps=3, exog=data_test[exog_features])"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "bad7ac00",
   "metadata": {},
   "source": [
    "As with any other forecaster, the matrices used during model training can be created with `create_train_X_y`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "9358a828",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>lag_1</th>\n",
       "      <th>lag_2</th>\n",
       "      <th>lag_3</th>\n",
       "      <th>lag_4</th>\n",
       "      <th>lag_5</th>\n",
       "      <th>holiday</th>\n",
       "      <th>weather</th>\n",
       "      <th>temp</th>\n",
       "      <th>hum</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>date_time</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2011-01-01 05:00:00</th>\n",
       "      <td>1.0</td>\n",
       "      <td>13.0</td>\n",
       "      <td>32.0</td>\n",
       "      <td>40.0</td>\n",
       "      <td>16.0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>9.84</td>\n",
       "      <td>75.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2011-01-01 06:00:00</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>13.0</td>\n",
       "      <td>32.0</td>\n",
       "      <td>40.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>9.02</td>\n",
       "      <td>80.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2011-01-01 07:00:00</th>\n",
       "      <td>2.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>13.0</td>\n",
       "      <td>32.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8.20</td>\n",
       "      <td>86.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2011-01-01 08:00:00</th>\n",
       "      <td>3.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>13.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>9.84</td>\n",
       "      <td>75.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2011-01-01 09:00:00</th>\n",
       "      <td>8.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>13.12</td>\n",
       "      <td>76.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                     lag_1  lag_2  lag_3  lag_4  lag_5 holiday weather   temp  \\\n",
       "date_time                                                                       \n",
       "2011-01-01 05:00:00    1.0   13.0   32.0   40.0   16.0       0       1   9.84   \n",
       "2011-01-01 06:00:00    1.0    1.0   13.0   32.0   40.0       0       0   9.02   \n",
       "2011-01-01 07:00:00    2.0    1.0    1.0   13.0   32.0       0       0   8.20   \n",
       "2011-01-01 08:00:00    3.0    2.0    1.0    1.0   13.0       0       0   9.84   \n",
       "2011-01-01 09:00:00    8.0    3.0    2.0    1.0    1.0       0       0  13.12   \n",
       "\n",
       "                      hum  \n",
       "date_time                  \n",
       "2011-01-01 05:00:00  75.0  \n",
       "2011-01-01 06:00:00  80.0  \n",
       "2011-01-01 07:00:00  86.0  \n",
       "2011-01-01 08:00:00  75.0  \n",
       "2011-01-01 09:00:00  76.0  "
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create training matrices\n",
    "# ==============================================================================\n",
    "X_train, y_train = forecaster.create_train_X_y(\n",
    "                       y    = data.loc[:end_train, 'users'],\n",
    "                       exog = data.loc[:end_train, exog_features]\n",
    "                   )\n",
    "X_train.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "92221c58",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['holiday', 'weather']"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Print the features considered as categorical by the forecaster\n",
    "# ==============================================================================\n",
    "estimator = forecaster.estimator\n",
    "cat_index = estimator.booster_.params.get('categorical_column')\n",
    "if cat_index is not None:\n",
    "    features_in_model = estimator.booster_.feature_name()\n",
    "    cat_features_in_model = [features_in_model[i] for i in cat_index]\n",
    "    \n",
    "cat_features_in_model"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "7a9aad4d",
   "metadata": {},
   "source": [
    "## Scikit-learn HistogramGradientBoosting"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "3e938a41",
   "metadata": {},
   "source": [
    "**Encoding the categories as integers and explicitly specifying the names of the categorical features**\n",
    "\n",
    "When creating a forecaster using `HistogramGradientBoosting`, the names of the categorical columns should be specified during the instantiation by passing them as a list to the `categorical_feature` argument."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "e455aea5",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Transformer: ordinal encoding\n",
    "# ==============================================================================\n",
    "# A ColumnTransformer is used to transform categorical (not numerical) features\n",
    "# using ordinal encoding. Numeric features are left untouched. Missing values\n",
    "# are coded as -1. If a new category is found in the test set, it is encoded\n",
    "# as -1.\n",
    "categorical_features = data.select_dtypes(exclude=[np.number]).columns.tolist()\n",
    "transformer_exog = make_column_transformer(\n",
    "                       (\n",
    "                           OrdinalEncoder(\n",
    "                               dtype=int,\n",
    "                               handle_unknown=\"use_encoded_value\",\n",
    "                               unknown_value=-1,\n",
    "                               encoded_missing_value=-1\n",
    "                           ),\n",
    "                           categorical_features\n",
    "                       ),\n",
    "                       remainder=\"passthrough\",\n",
    "                       verbose_feature_names_out=False,\n",
    "                   ).set_output(transform=\"pandas\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "cf6ff94b",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create and fit forecaster indicating the categorical features\n",
    "# ==============================================================================\n",
    "exog_features = ['holiday', 'weather', 'temp', 'hum']\n",
    "forecaster = ForecasterRecursive(\n",
    "                 estimator = HistGradientBoostingRegressor(\n",
    "                                 categorical_features = categorical_features,\n",
    "                                 random_state = 963\n",
    "                             ),\n",
    "                 lags = 5,\n",
    "                 transformer_exog = transformer_exog\n",
    "             )\n",
    "            \n",
    "forecaster.fit(\n",
    "    y    = data.loc[:end_train, 'users'],\n",
    "    exog = data.loc[:end_train, exog_features]\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "8d04c67f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2012-08-01 00:00:00    99.185547\n",
       "2012-08-01 01:00:00    71.914255\n",
       "2012-08-01 02:00:00    43.342723\n",
       "Freq: h, Name: pred, dtype: float64"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Predictions\n",
    "# ==============================================================================\n",
    "forecaster.predict(steps=3, exog=data_test[exog_features])"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "6f5c4ebd",
   "metadata": {},
   "source": [
    "`HistGradientBoostingRegressor` stores a boolean mask indicating which features were considered categorical. It will be `None` if there are no categorical features."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "54ea8937",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['holiday', 'weather'], dtype=object)"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Print the features considered as categorical by the forecaster\n",
    "# ==============================================================================\n",
    "forecaster.estimator.feature_names_in_[forecaster.estimator.is_categorical_]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "36d13cd5",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Feature 'holiday' categories and codes:\n",
      "  0: 0\n",
      "  1: 1\n",
      "Feature 'weather' categories and codes:\n",
      "  clear: 0\n",
      "  mist: 1\n",
      "  rain: 2\n"
     ]
    }
   ],
   "source": [
    "# Show the encoding applied to the categorical features\n",
    "# ==============================================================================\n",
    "ordinal_encoder = transformer_exog.named_transformers_['ordinalencoder']\n",
    "for feature, cats in zip(categorical_features, ordinal_encoder.categories_):\n",
    "    print(f\"Feature '{feature}' categories and codes:\")\n",
    "    for code, category in enumerate(cats):\n",
    "        print(f\"  {category}: {code}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0aa4ab62",
   "metadata": {},
   "source": [
    "**Allow the model to automatically detect categorical features**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "9567b3e3",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Transformer: ordinal encoding and cast to category type\n",
    "# ==============================================================================\n",
    "# A ColumnTransformer is used to transform categorical (not numerical) features\n",
    "# using ordinal encoding. Numeric features are left untouched. Missing values\n",
    "# are coded as -1. If a new category is found in the test set, it is encoded\n",
    "# as -1. After encoding, the features are converted back to category type so that \n",
    "# they can be identified as categorical features by the estimator.\n",
    "\n",
    "pipeline_categorical = make_pipeline(\n",
    "                           OrdinalEncoder(\n",
    "                               dtype=int,\n",
    "                               handle_unknown=\"use_encoded_value\",\n",
    "                               unknown_value=-1,\n",
    "                               encoded_missing_value=-1\n",
    "                           ),\n",
    "                           FunctionTransformer(\n",
    "                               func=lambda x: x.astype('category'),\n",
    "                               feature_names_out= 'one-to-one'\n",
    "                           )\n",
    "                       )\n",
    "\n",
    "transformer_exog = make_column_transformer(\n",
    "                       (\n",
    "                           pipeline_categorical,\n",
    "                           make_column_selector(dtype_exclude=np.number)\n",
    "                       ),\n",
    "                       remainder=\"passthrough\",\n",
    "                       verbose_feature_names_out=False,\n",
    "                   ).set_output(transform=\"pandas\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "24090f42",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create and fit forecaster with automatic detection of categorical features\n",
    "# ==============================================================================\n",
    "exog_features = ['holiday', 'weather', 'temp', 'hum']\n",
    "forecaster = ForecasterRecursive(\n",
    "                 estimator        = HistGradientBoostingRegressor(random_state=963, categorical_features='from_dtype'),\n",
    "                 lags             = 5,\n",
    "                 transformer_exog = transformer_exog,\n",
    "             )\n",
    "            \n",
    "forecaster.fit(\n",
    "    y    = data.loc[:end_train, 'users'],\n",
    "    exog = data.loc[:end_train, exog_features]\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "7dfcc83b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['holiday', 'weather'], dtype=object)"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Print the features considered as categorical by the forecaster\n",
    "# ==============================================================================\n",
    "forecaster.estimator.feature_names_in_[forecaster.estimator.is_categorical_]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "b894c62c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2012-08-01 00:00:00    99.185547\n",
       "2012-08-01 01:00:00    71.914255\n",
       "2012-08-01 02:00:00    43.342723\n",
       "Freq: h, Name: pred, dtype: float64"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Predictions\n",
    "# ==============================================================================\n",
    "forecaster.predict(steps=3, exog=data_test[exog_features])"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "9b9a0b27",
   "metadata": {},
   "source": [
    "## XGBoost"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "7e0a2419",
   "metadata": {},
   "source": [
    "**Encoding the categories as integers and explicitly specifying the names of the categorical features**\n",
    "\n",
    "At the time of writing, the `XGBRegressor` module does not provide an option to specify the names of categorical features. Instead, the feature types are specified by passing a list of strings to the `feature_types` argument, where 'c' denotes categorical and 'q' numeric features. The `enable_categorical` argument must also be set to `True`.\n",
    "\n",
    "Determining the positions of each column to create a list of feature types can be a challenging task. The shape of the data matrix depends on two factors, the number of lags used and the transformations applied to the exogenous variables. However, there is a workaround to this problem. First, create a forecaster without specifying the `feature_types` argument. Next, the `create_train_X_y` method can be used with a small sample of data to determine the position of each feature. Once the position of each feature has been determined, the `set_params()` method can be used to specify the values of `feature_types`. By following this approach it is possible to ensure that the feature types are correctly specified, thus avoiding any errors that may occur due to incorrect specification."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "35230362",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Transformer: ordinal encoding\n",
    "# ==============================================================================\n",
    "# A ColumnTransformer is used to transform categorical (no numerical) features\n",
    "# using ordinal encoding. Numeric features are left untouched. Missing values\n",
    "# are coded as -1. If a new category is found in the test set, it is encoded\n",
    "# as -1.\n",
    "categorical_features = data.select_dtypes(exclude=[np.number]).columns.tolist()\n",
    "transformer_exog = make_column_transformer(\n",
    "                       (\n",
    "                           OrdinalEncoder(\n",
    "                               dtype=int,\n",
    "                               handle_unknown=\"use_encoded_value\",\n",
    "                               unknown_value=-1,\n",
    "                               encoded_missing_value=-1\n",
    "                           ),\n",
    "                           categorical_features\n",
    "                       ),\n",
    "                       remainder=\"passthrough\",\n",
    "                       verbose_feature_names_out=False,\n",
    "                   ).set_output(transform=\"pandas\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "b1fdd54d",
   "metadata": {},
   "source": [
    "A forecaster is created without specifying the `feature_types` argument."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "48f5b3a7",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create forecaster\n",
    "# ==============================================================================\n",
    "exog_features = ['holiday', 'weather', 'temp', 'hum']\n",
    "forecaster = ForecasterRecursive(\n",
    "                 estimator = XGBRegressor(\n",
    "                                 tree_method        = 'hist',\n",
    "                                 random_state       = 12345,\n",
    "                                 enable_categorical = True,\n",
    "                             ),\n",
    "                 lags = 5,\n",
    "                 transformer_exog = transformer_exog\n",
    "             )"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "fa30c3ac",
   "metadata": {},
   "source": [
    "Once the forecaster is instantiated, its `create_train_X_y()` method is used to generate the training matrices that allow the user to identify the positions of the variables."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "642c3dd2",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>lag_1</th>\n",
       "      <th>lag_2</th>\n",
       "      <th>lag_3</th>\n",
       "      <th>lag_4</th>\n",
       "      <th>lag_5</th>\n",
       "      <th>holiday</th>\n",
       "      <th>weather</th>\n",
       "      <th>temp</th>\n",
       "      <th>hum</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>date_time</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2011-01-01 05:00:00</th>\n",
       "      <td>1.0</td>\n",
       "      <td>13.0</td>\n",
       "      <td>32.0</td>\n",
       "      <td>40.0</td>\n",
       "      <td>16.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>9.84</td>\n",
       "      <td>75.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2011-01-01 06:00:00</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>13.0</td>\n",
       "      <td>32.0</td>\n",
       "      <td>40.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>9.02</td>\n",
       "      <td>80.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                     lag_1  lag_2  lag_3  lag_4  lag_5  holiday  weather  \\\n",
       "date_time                                                                  \n",
       "2011-01-01 05:00:00    1.0   13.0   32.0   40.0   16.0      0.0      1.0   \n",
       "2011-01-01 06:00:00    1.0    1.0   13.0   32.0   40.0      0.0      0.0   \n",
       "\n",
       "                     temp   hum  \n",
       "date_time                        \n",
       "2011-01-01 05:00:00  9.84  75.0  \n",
       "2011-01-01 06:00:00  9.02  80.0  "
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create training matrices using a sample of the training data\n",
    "# ==============================================================================\n",
    "X_train, y_train = forecaster.create_train_X_y(\n",
    "                       y    = data.loc[:end_train, 'users'][:10],\n",
    "                       exog = data.loc[:end_train, exog_features][:10]\n",
    "                   )\n",
    "X_train.head(2)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "a3b81f8a",
   "metadata": {},
   "source": [
    "Create a list to identify which columns in the training matrix are numeric ('q') and categorical ('c')."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "acc6b116",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['q', 'q', 'q', 'q', 'q', 'c', 'c', 'q', 'q']"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "feature_types = [\n",
    "    \"c\" if X_train[col].dtype.name in [\"object\", \"category\"] or col in categorical_features\n",
    "    else \"q\"\n",
    "    for col in X_train.columns\n",
    "]\n",
    "feature_types"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "7d9a5664",
   "metadata": {},
   "source": [
    "Update the estimator parameters using the forecaster's `set_params` method and fit."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "02302bf5",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Update estimator parameters\n",
    "# ==============================================================================\n",
    "forecaster.set_params({'feature_types': feature_types})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "85d8254b",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Fit forecaster\n",
    "# ==============================================================================\n",
    "forecaster.fit(\n",
    "    y    = data.loc[:end_train, 'users'],\n",
    "    exog = data.loc[:end_train, exog_features]\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "id": "f36cab79",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2012-08-01 00:00:00    82.127357\n",
       "2012-08-01 01:00:00    45.740013\n",
       "2012-08-01 02:00:00    23.580956\n",
       "Freq: h, Name: pred, dtype: float64"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Predictions\n",
    "# ==============================================================================\n",
    "forecaster.predict(steps=3, exog=data_test[exog_features])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "id": "9b5b6d81",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['holiday', 'weather'], dtype='<U7')"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Print the features considered as categorical by the forecaster\n",
    "# ==============================================================================\n",
    "feature_types = np.array(forecaster.estimator.get_booster().feature_types)\n",
    "features_in_model = np.array(forecaster.estimator.get_booster().feature_names)\n",
    "features_in_model[feature_types == 'c']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "id": "bf578d09",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Feature 'holiday' categories and codes:\n",
      "  0: 0\n",
      "  1: 1\n",
      "Feature 'weather' categories and codes:\n",
      "  clear: 0\n",
      "  mist: 1\n",
      "  rain: 2\n"
     ]
    }
   ],
   "source": [
    "# Show the encoding applied to the categorical features\n",
    "# ==============================================================================\n",
    "ordinal_encoder = transformer_exog.named_transformers_['ordinalencoder']\n",
    "for feature, cats in zip(categorical_features, ordinal_encoder.categories_):\n",
    "    print(f\"Feature '{feature}' categories and codes:\")\n",
    "    for code, category in enumerate(cats):\n",
    "        print(f\"  {category}: {code}\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "65824211",
   "metadata": {},
   "source": [
    "**Allow the model to automatically detect categorical features**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "id": "b0be2171",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Transformer: ordinal encoding and cast to category type\n",
    "# ==============================================================================\n",
    "# A ColumnTransformer is used to transform categorical (no numerical) features\n",
    "# using ordinal encoding. Numeric features are left untouched. Missing values\n",
    "# are coded as -1. If a new category is found in the test set, it is encoded\n",
    "# as -1. After the encoding, the features are converted back to category type so \n",
    "# that they can be identified as categorical features by the estimator.\n",
    "\n",
    "pipeline_categorical = make_pipeline(\n",
    "                           OrdinalEncoder(\n",
    "                               dtype=int,\n",
    "                               handle_unknown=\"use_encoded_value\",\n",
    "                               unknown_value=-1,\n",
    "                               encoded_missing_value=-1\n",
    "                           ),\n",
    "                           FunctionTransformer(\n",
    "                               func=lambda x: x.astype('category'),\n",
    "                               feature_names_out= 'one-to-one'\n",
    "                           )\n",
    "                       )\n",
    "\n",
    "transformer_exog = make_column_transformer(\n",
    "                       (\n",
    "                           pipeline_categorical,\n",
    "                           make_column_selector(dtype_exclude=np.number)\n",
    "                       ),\n",
    "                       remainder=\"passthrough\",\n",
    "                       verbose_feature_names_out=False,\n",
    "                   ).set_output(transform=\"pandas\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "id": "5d1051d4",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create and fit forecaster\n",
    "# ==============================================================================\n",
    "exog_features = ['holiday', 'weather', 'temp', 'hum']\n",
    "forecaster = ForecasterRecursive(\n",
    "                 estimator = XGBRegressor(\n",
    "                                 enable_categorical=True,\n",
    "                                 tree_method='hist',\n",
    "                                 random_state=963\n",
    "                             ),\n",
    "                 lags = 5,\n",
    "                 transformer_exog = transformer_exog\n",
    "             )\n",
    "            \n",
    "forecaster.fit(\n",
    "    y    = data.loc[:end_train, 'users'],\n",
    "    exog = data.loc[:end_train, exog_features]\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "id": "3d915fdb",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2012-08-01 00:00:00    82.127357\n",
       "2012-08-01 01:00:00    45.740013\n",
       "2012-08-01 02:00:00    23.580956\n",
       "Freq: h, Name: pred, dtype: float64"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Predictions\n",
    "# ==============================================================================\n",
    "forecaster.predict(steps=3, exog=data_test[exog_features])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "id": "2ad60f48",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['holiday', 'weather'], dtype='<U7')"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Print the features considered as categorical by the forecaster\n",
    "# ==============================================================================\n",
    "feature_types = np.array(forecaster.estimator.get_booster().feature_types)\n",
    "features_in_model = np.array(forecaster.estimator.get_booster().feature_names)\n",
    "features_in_model[feature_types == 'c']"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "b2e48ce0",
   "metadata": {},
   "source": [
    "## CatBoost\n",
    "\n",
    "Unfortunately, the current version of skforecast is not compatible with CatBoost's built-in handling of categorical features. The issue arises because CatBoost only accepts categorical features as integers, while skforecast converts input data to floats for faster computation using numpy arrays in the internal prediction process. If a CatBoost model is required, an external encoder should be used for the categorical variables."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "skforecast_py12",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
